Search for: All records

Physical therapy is often essential for complete recovery after injury. However, a significant population of patients fail to adhere to prescribed exercise regimens. Lack of motivation and inconsistent in-person visits to physical therapy are major contributing factors to suboptimal exercise adherence, slowing the recovery process. With the advancement of virtual reality (VR), researchers have developed remote virtual rehabilitation systems with sensors such as inertial measurement units. A functional garment with an integrated wearable sensor can also be used for real-time sensory feedback in VR-based therapeutic exercise and offers affordable remote rehabilitation to patients. Sensors integrated into wearable garments offer the potential for a quantitative range of motion measurements during VR rehabilitation. In this research, we developed and validated a carbon nanocomposite-coated knit fabric-based sensor worn on a compression sleeve that can be integrated with upper-extremity virtual rehabilitation systems. The sensor was created by coating a commercially available weft knitted fabric consisting of polyester, nylon, and elastane fibers. A thin carbon nanotube composite coating applied to the fibers makes the fabric electrically conductive and functions as a piezoresistive sensor. The nanocomposite sensor, which is soft to the touch and breathable, demonstrated high sensitivity to stretching deformations, with an average gauge factor of ~35 in the warp direction of the fabric sensor. Multiple tests are performed with a Kinarm end point robot to validate the sensor for repeatable response with a change in elbow joint angle. A task was also created in a VR environment and replicated by the Kinarm. The wearable sensor can measure the change in elbow angle with more than 90% accuracy while performing these tasks, and the sensor shows a proportional resistance change with varying joint angles while performing different exercises. The potential use of wearable sensors in at-home virtual therapy/exercise was demonstrated using a Meta Quest 2 VR system with a virtual exercise program to show the potential for at-home measurements. 

Anatomy education is an indispensable part of medical training, but traditional methods face challenges like limited resources for dissection in large classes and difficulties understanding 2D anatomy in textbooks. Advanced technologies, such as 3D visualization and augmented reality (AR), are transforming anatomy learning. This paper presents two in-house solutions that use handheld tablets or screen-based AR to visualize 3D anatomy models with informative labels and in-situ visualizations of the muscle anatomy. To assess these tools, a user study of muscle anatomy education involved 236 premedical students in dyadic teams, with results showing that the tablet-based 3D visualization and screen-based AR tools led to significantly higher learning experience scores than traditional textbook. While knowledge retention didn’t differ significantly, ethnographic and gender analysis showed that male students generally reported more positive learning experiences than female students. This study discusses the implications for anatomy and medical education, highlighting the potential of these innovative learning tools considering gender and team dynamics in body painting anatomy learning interventions. 

For serious games on education, understanding the effectiveness of different learning methods in influencing cognitive processes remains a significant challenge. In particular, limited research addresses the comparative effectiveness of serious games and videos in analyzing brain behavior for graph structure learning, which is an important part of the Science, Technology, Engineering, Math, and Computing (STEM+C) disciplinary education. This study investigates the impact of serious games on graph structure learning. For this, we compared our in-house game-based learning (GBL) and video-based learning (VBL) methodologies by evaluating their effectiveness on cognitive processes by oxygenated hemoglobin levels using functional near-infrared spectroscopy (fNIRS). We conducted a 2×1 between-subjects preliminary study with twelve participants, involving two conditions: game and video. Both groups received equivalent content related to the basic structure of a graph, with comparable session lengths. The game group interacted with a quiz-based game, while the video group watched a pre-recorded video. The fNIRS was employed to capture cerebral signals from the prefrontal cortex, and participants completed pre- and post-questionnaires capturing user experience and knowledge gain. In our study, we noted that the mean levels of oxygenated hemoglobin (delta HbO) were higher in the GBL group, suggesting the potential enhanced cognitive involvement. Our results show that the lateral prefrontal cortex (LPFC) has greater hemodynamic activity during the learning period. Moreover, knowledge gain analysis showed an increase in mean score in the GBL group compared to the VBL group. Although we did not observe statistically significant changes due to participant variability and sample size, this preliminary work contributes to understanding how GBL and VBL impact cognitive processes, providing insights for enhanced instructional design and educational game development. Additionally, it emphasizes the necessity for further investigation into the impact of GBL on cognitive engagement and learning outcomes. 

Virtual reality (VR) and interactive 3D visualization systems have enhanced educational experiences and environments, particularly in complicated subjects such as anatomy education. VR-based systems surpass the potential limitations of traditional training approaches in facilitating interactive engagement among students. However, research on embodied virtual assistants that leverage generative artificial intelligence (AI) and verbal communication in the anatomy education context is underrepresented. In this work, we introduce a VR environment with a generative AI-embodied virtual assistant to support participants in responding to varying cognitive complexity anatomy questions and enable verbal communication. We assessed the technical efficacy and usability of the proposed environment in a pilot user study with 16 participants. We conducted a within-subject design for virtual assistant configuration (avatar- and screen-based), with two levels of cognitive complexity (knowledge- and analysis-based). The results reveal a significant difference in the scores obtained from knowledge- and analysis-based questions in relation to avatar configuration. Moreover, results provide insights into usability, cognitive task load, and the sense of presence in the proposed virtual assistant configurations. Our environment and results of the pilot study offer potential benefits and future research directions beyond medical education, using generative AI and embodied virtual agents as customized virtual conversational assistants.